Papers with inner representations
Think in Sentences: Explicit Sentence Boundaries Enhance Language Model’s Capabilities (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on dummy tokens but fail to leverage the inherent sentence-level structure of natural language. |
| Approach: | They propose a method that inserts delimiters at sentence boundaries to enhance large language models' capabilities. |
| Outcome: | The proposed method improves performance on 7B LLMs to 600B Deepseek-V3 with 7.7% gains on GSM8k and 12.5% on DROP. |
Debiasing Methods in Natural Language Understanding Make Bias More Accessible (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent debiasing methods in natural language understanding improve performance on out-of-distribution datasets by pressuring models into making unbiased predictions. |
| Approach: | They propose a general probing-based framework that allows for post-hoc interpretation of biases in language models and use an information-theoretic approach to measure the extractability of certain biase . |
| Outcome: | The proposed framework allows for post-hoc interpretation of biases in language models and measures the extractability of certain biase . |
Where Confabulation Lives: Latent Feature Discovery in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite advances in large language models, hallucination remains a critical failure mode . despite these advances, LLMs are prone to hallucinic outputs that contain illusory information presented as if it were factual or grounded in reality. |
| Approach: | They propose to isolate and analyze confabulation, a foundational aspect of hallucination, where the model fabricates facts about unknown entities. |
| Outcome: | The proposed method reveals that the model can fabricate facts with minimal disruption, shedding light on the inner representations that drive factual and non-factual output. |